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to schools implementing these new curricula. The core of the 
resulting 15-unit TLTG science program is it. c interactive video-based 
instruction. Robbins Secondary School was selected as a site for the 
TLTG physical science program to determine if interactive video 
instruction could also enhance the learning of at-risk students. An 
evaluation of the program found that most students evinced a positive 
attitude toward the technology provided, felt comfortable using the 
computer assisted instructional system, enjoyed learning science that 
way, and felt they were learning the material better using this 
technology. Survey results also indicated that, despite positive 
student attitudes, they did not express an interest in continuing 
science classes. Finally, students' grade equivalent scores on t>ie 
TAP Science Test were lower than typical for their grade and lower 
than those of other students at Robbins* A copy of the attitude 
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Executive Summary' 
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Program Description 



The Texas Learning Technology 
G^up (TLTG) is a nonprofit organiza- 
tion formed in 1985 by the Texas 
Association of School Boards (TASB), 
the National Science Center Founda- 
tion (NSCF), and 12 Texas school 
districts including Austin Independent 
School District (AISD). Believing that 
science education had become charac- 
terized by a shortage of qualified 
physical science teachers, a high 
failure rate, a lack of student motiva- 
tion, and a decreasing number of 
students enrolling in advanced science 
courses, the organization established a 
three-fold mission: 

a) To develop high-quality curricu- 
lum programs that integrate new 
technologies into instructional 
delivery systems, 

b) To evaluate the effectiveness of 
technology-based curricula, and 

c) To train teachers in the use of 
new technologies and provide 
support to schools implementing 
these new curricula. 



The core of the resulting 15-unit 
TLTG physical science program is its 
interactive video-based instruction. 
Goals of the TLTG program include 
increasing students' in-depth under- 
standing of physical science concepts, 
showing the relevance of physical 
science to daily life, and preparing 
students for academic and professional 
advancement in the sciences. 
Implemented at a number of Texas 
schools, pilot studies indicated that the 
TLTG physical science program was 
successful in producing significant 
gains in achievement and in increasing 
positive attitudes toward science for 
students identified as having low 
verbal and low quantitative ability. 

Robbins Secondary School was 
selected as a site for the TLTG 
physical science program to determine 
if interactive video instruction could 
also enhance the learning of at-risk 
students. Twenty-three students 
participated in the program during the 
1990-91 school year. 



Major Findings 

1. Most of the TLTG students 
evinced a positive attitude 
toward the technology provided. 
In response to a survey, a 
majority reported feeling 

con "ortable using the computer 
and \ deodisc system, enjoyed 
using ihem in class, thought 
they were a good way to learn 
science material, and would like 
to use them in other classes. 
More than two thirds also 
believed they learn science 
better using this technology 
(pp. 7-8). 

2. The survey results also indicate 
that although the students' 
attitudes toward science were 
mostly positive, they did not 
express an interest in continuing 
to take science classes or in 
pursuing science outside the 
classroom (pp. 8-9). 

3. TLTG students' grade equiva- 
lent scores on the TAP Science 
Test are lower than typical for 
their grade, and lower than 
those of other students at 
Robbins (pp. 5-7). 



A copy of the full report for which this is the Executive Summary is available as 

Publication Number 9034 
from: 

Austin Independent School District 
Office of Research and Evaluation 
1111 West 6th Street 
Austin, Texas 78703-5399 
(512)499-1724 
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TECHNOLOGY fi AT-RISK STUDENTS 5 
THE TLTO SCIENCE PROJECT AT ROBBINS 
FINAL REPORT 

PROGRAM DESCRIPTION 

The TLTG Program 

The Texas Learning Technology Group (TLTG) is a nonprofit 
organization formed in 1985 by the Texas Association of School 
Boards (TASB) , the National Science Center Foundation (NSCF) , and 
12 Texas school districts (including Austin ISD) . Believing that 
science education had become characterized by a shortage of 
qualified physical science teachers, a high failure rate, a lack 
of student motivation, and a decreasing number of students 
enrolling in advanced science courses, th« organization 
established a three-fold mission: 

o To develop high-quality curriculum programs that 

integrate new technologies into instructional delivery 
systems , 

o To evaluate the effectiveness of technology-based 

curricula, and 
o To train teachers in the use of new technologies and 

provide support to schools implementing these new 

curricula (1988-89 Field Test Results of TLTG Physical 

Science, 1989). 

The core of the resulting 15-unit TLTG physical science program 
is its interactive video-based instruction. Goals of the TLTG 
program are to: 

o Increase students' in-depth understanding of physical 

science concepts, 
o Show the relevance of physical science to daily life, 

and 

o Prepare students for academic and professional 
advancement in the sciences. 

Robbins Secondary School 

The TLTG physical science program was first implemented at a 
number of Texas schools including Austin's Anderson High School. 
Pilot studies indicated that the program was successful in 
producing significant gains in achievement and in increasing 
positive attitudes toward science for students identified as 
having low verbal and low quantitative ability. The next step 
was to determine if interactive video instruction could enhance 
the learning of at-risk students as well. 

Robbins Secondary School is an alternative school offering a 
self-paced academic program for students having difficulties in a 
traditional school setting. While AISD's 1990-91 high school at- 
risk population was 49%, the at-risk population at Robbins in 
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grades 9-12 was a staggering 91%. The Robbins alternative 
education plan uses a contract system in which students work 
individually at their own pace to accomplish what is necessary 
for each course. Keeping students motivated to stay in school 
and complete their studies is a high priority. 

The objectives of the TLTG program implemented at Robbins were 
to: 

o Decrease dropout rates at Robbins, 

o Increase students' science achievement, 

o Increase students' positive attitudes towards 

science and technology, 
o Increase students' knowledge about technology 

in their environment, and 
o Increase teachers' use of technology-oriented 

instructional programs. 



PROGRAM IMPLEMENTATION 

Teacher Training 

The Robbins science teacher attended two training sessions before 
introducing the TLTG program on her campus in the fall of 1990. 
The first one took place in February 1990 and involved everyone 
using the system. The second was held in August at the TLTG 
office. During this one-on-one session, the teacher received 
instruction on the software and how to present it to the class. 

Equipment and Software 

TLTG provided one teacher station and two student stations along 
with two semesters of physical science courseware for the 
program. The equipment included one IBM PS/2 Model 60 computer, 
two IBM PS/2 Model 50Z computers, two IBM color monitors, one 
Sony PVM-2539 color monitor, three Sony LDP-1200 laser disc 
players, and three Sony RM2001 remote control units. The 
curriculum included 14 videodiscs, accompanying software, 
chemistry and physics resource guides, chemistry and physics 
student guides, and assessment program. 

The software arrived on the first day of the fall semester but 
because of some initial debugging, the students did not begin 
working with it until a few weeks later. 

Class Structure 

An alternative school, Robbins structures its classes in a unique 
way. All the students in the first-period science class are not 
enrolled in the same course. As many as nine students or as few 
as one are registered for Introduction to Physical Science during 
any given period. The other courses offered by the teacher 
include Physical Science, Introduction to Biology, Astronomy, and 
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Chemistry. Because Robbins is a self -paced school, students work 
on their individual contracts at their own pace. Instead of 
lecturing to the class as a whole, the teacher assists students 
individually during the class period. 

All assignments for each class are outlined in a contract, and 
all courses at Robbins use contracts. For each lesson, the TLTG 
contracts require students to watch a video presentation (in lieu 
of reading a chapter or section in a textbook) and complete a 
practice set or worksheet. Students then either watch another 
video presentation, complete a lab, or do an interactive 
assignment on the computer. Not all students use the videodisc 
system every day, but the systems are used by someone every day. 
The textbook functions mostly as a reference, and the teacher has 
designed a chart that correlates all TLTG assignments with 
chapters in the physcial science textbook. 

All science students, regardless of the course in which they were 
enrolled, were introduced to the system. Every contract used the 
computer and videodisc system for the first three assignments 
which included lessons on safety and the scientific method. The 
students enrolled in the Introduction to Physical Science course 
were trained individually or in small groups to turn the system 
on, pull up the materials, and read the contracts. 

STUDENT CHARACTERISTICS 

A total of 23 students wa«* enrolled in the Introduction to 
Physical Science course ciuring the 1990-91 school year. Ten 
students took the course in the fall (six of these students 
continued with the second semester in the spring) , and 15 more 
students enrolled during the spring. 

Of the 23 students served: 

o The mean age was 17 years, eight months; the youngest 

student was 16 years, four months, and the oldest 

student was 19 years, five months; 
o 52% (12) were male, and 48% (11) were female; 
o 65% (15) were Black, 30% (7) were Hispanic, and 

4% (1) were Other; 
o 44% (10) of the students were 9th graders, 44% (10) 

were 10th graders, 9% (2) were 11th graders, and 

4% (1) were 12th graders; 
o 43% (10) were from low- income families. 



At-Riskness 

At-risk students are those with certain characteristics which 
increase the likelihood that they will drop out of school. In 
order to identify and track these students, AISD operational i zed 
State at-risk criteria and developed a list of 22 at-risk 
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categories (see ORE Publication Number 88.36). The categories 
include: 

o Overage for grade; that is, a student is two or 

more years older than expected for the grade level 

as of September 1; 
o scoring two or more years below grade level on a 

norm-referenced standardized achievement test; 
o Failing two or more courses during one semester; 
o Failing any section of the most recently administered 

state-mandated minimum skills test; 
o Combinations of the above categories. 

In 1990-91, every one of the TLTG students was identified as 
being at-risk: 

o All but one (96%) were overage for grade. 

o All students scored two or more years below grade level 

on at least one section of the Tests of Achievement and 

Proficiency; 83% scored two years or more below grade 

level on two sections, 
o Most (16, 70%) failed two or more courses in one 

semester. 

o Most (17, 74%) failed one or more sections of the last 
Texas Educational Assessment of Minimum Skills (TEAMS) 
taken. 

It is important to note that these at-risk statistics for the 
TLTG students are higher than for the Robbins Secondary School 
population overall. 



PROGRAM RESULTS 

Did the dropout rate at Robbins decrease? 

Yes. The dropout rate at Robbins in 1990-91 through the fifth 
six weeks for grades 9-12 was 19.4%, which is 89.8% of the 
dropout rate for 1989-90. During the year of program 
implementation, 13% (3) of the TLTG students transferred to 
another school in the District, and 4% (1) dropped out. 

Did TLTG students' science achievement increase? 

Although the TLTG students made some achievement gains, their 
scores were lower than typical for their grade level, and not as 
large as other Robbins students. 

Achievement was measured by comparing program students' scores on 
the Science Test of the Tests of Achievement and Proficiency 
(TAP) with their scores from previous years and with the scores 
of other high school students at Robbins and in the District (see 
Figures 1-4) . It should be noted that other comparisons are 
possible, including comparisons to at-risk students at other 
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schools taking Introduction to Physical Science. The comparisons 
shown here are reasonable since TLTG students are Robbins 
students, and other Robbins students have taken other science 
courses, and therefore should also be making some achievement 
gains in science. 

To compare the achievement of TLTG students and Robbins students 
in science, a straightforward comparison of the median TAP 
Science scores of these students was ^irst considered. This 
comparison was not satisfactory, however, because students at 
different high school grade levels take different levels of th« 
TAP, and the number of TLTG students at some grade levels was too 
small to compute a median. A single group score representative 
of the TLTG students as a whole was preferable, but computing a 
single score across grade levels, effectively collapsing across 
test levels, is technically dubious. Another measure reflective 
of group achievement was needed for both the TLTG and Robbins 
students . 

Two comparative achievement measures were found: 

1) The average gain on the TAP Science Test from spring 
1990 to spring 1991, and 

2) The average distance from typical (average) grade 
equivalent on the spring 1991 TAP Science Test. 

The first measure was calculated by computing the spring-to- 
spring gain (in grade equivalents) for each student, summing the 
gains for all students, and dividing by the number of students 
with scores for both years. 

The second measure was derived as follows. The average grade 
equivalent score at any grade level is the number of years 
corresponding to the grade level and the number of months 
corresponding to the month in which the test was administered. 
In AISD, achievement tests are administered in April, the eighth 
month of the school year; therefore, the average grade equivalent 
for ninth graders is, for example, 9.8, for tenth graders, 10.8, 
and so on. For each student, the difference between the 
student's spring 1991 grade equivalent score and the average 
score expected for a student in that grade level was computed. 
Differences were summed for all students, and the resulting total 
was divided by the number of students, 

As shown in Figure 1, the median grade equivalents for TLTG 
students on the TAP science Test in spring 1991 were lower than 
the scores of all Robbins students for the last three years. 

Medians could not be calculated for grades 11 and 12 because too 
few students enrolled in TLTG in those grades. In all cases, the 
number of TLTG students for each grade level is small, and a 
median for these groups should not be considered a reliable 
measure of achievement. 
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FIGURE 1 
TAP SCIENCE ACHIEVEMENT 
TLTG STUDENTS VS. ROBBINS 

vtjiuue TL/Io Ri'DBlNS ROB DINS ROBBINS 

Level 1991 1991 1990 1989 



9 

GE 7.25* 8.95 7.93 9.31 

%ile 16* 39 24 44 

N 8 77 61 69 



GE 

%ile 

W 



10 



7.9* 
17* 

9 



8.44 
21 
43 



8.68 
24 
34 



04 
25 
25 



■tile 
N 



11 



NA 
NA 
1 



9.05* 
22* 
18 



9.80 
30 
20 



11.00* 
40* 

8 



12 

GE NA 10.08* 9. 00* 

%ile NA 26* 16* 

N 1 12 9 



NA'Not available. There arc too few students to calculate a median. 
* A ivKitan r.aiculated for a smell number c* students should not be considered as a 
relubte measu'c of a group's achievement. 

As snown in figure 2, TLTG students 9 g *\Za equivalent scores on 
the TAP science Test are lover than exp*^~ed for their grade and 

lower than other students at Robbins and in the District* 



9.45* 
21* 
14 



FIGURE 2 
TAP SCIENCE ACHIEVEMENT 1991 
TLTG STUDENTS COMPARED TO ROBBINS AND AISD 

Qrad« TLTG R0BBXN8 AISD 

L«V«1 1991 1991 1991 



9 

GE 7.25* 8.95 10.69 

lile 16* 3? 60 

N 8 77 3779 



10 

GE 7.90* 8.44 12.85 

%ile 17* 21 67 

N 9 43 3050 



11 

GE NA 9.05* 13.54 

%ile NA 22* 65 

N 1 18 2517 



12 

GE NA 10.08* 13.28 

%ile NA 26* 54 

N 1 12 2232 



HA=K<?t available. There ire too few students to calculate a Median. 

* * median calculated for a small number of students should not be considered as a 
reliable measure of a group's achievement. 
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The average gain on TAP Science from spring 1990 to spring 1991 
for TLTG students and Robbins students is depicted in Figure 3. 
TLTG students' gain was not as large as that of other Robbins 
students. 

FIGURE 3 

AVERAGE GRADE EQUIVALENT GAIN ON TAP SCIENCE TEST 1990-91 
TLTG COMPARED TO OTHER ROBBINS STUDENTS 



AVERAGE GRADE 
N EQUIVALENT GAIN 



TLTG 


19 


0. 


54 


ROBBINS 


135 


0. 


61 



Figure 4 shows that the average distance of TLTG students' grade 
equivalent scores from expected grade- level scores is much 
greater than that of other Robbins students. 

FIGURE 4 

AVERAGE DIFFERENCE OF GRADE EQUIVALENT 

SCORE FROM GRADE LEVEL 1990-91 
TLTG COMPARED TO OTHER ROBBINS STUDENTS 

AVERAGE DISTANCE 
N FROM GRADE LEVEL 



TLTG 15 -2.97 



ROBBINS 71 -0.88 



Student Attitude Survey 

In May 1991, a survey assessing previous experience with 
technology and attitude toward scienco and t Sinology was 
administered to students enrolled in the Introduction to Physical 
Science course during the spring semester. Thirteen surveys were 
completed. The results for all survey questions are shown in the 
Appendix. 

Before taking this course, the TLTC students had limited exposure 
to computer and videodisc technology. The responses to the 
survey show that almost two thirds (62%) of the students had 
never or almost never used a computer, and all but one (92%) had 
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never used a videodisc system. Three fourths (77%) said that 
they are comfortable using computers, and the remainder (23%) 
were neutral. Two thirds (69%) said that they are comfortable 
using a videodisc system, 23% were neutral, and one respondent 
(8%) was not comfortable. Almost all students (92%) report that 
they enjoy using the computer, while two thirds (69%) said they 
enjoy using the videodisc system. 

Did positive attitudes toward science and technology increase? 

Most students evinced a positive attitude toward using the 
technology provided in that a majority felt comfortable using the 
computer and videodisc system, enjoyed using them in class, 
thought they were a good way to learn science material, and would 
like to use them in other classes. Students' responses were 
generally more positive about computers than videodisc systems. 

Results to the attitude toward technology questions include: 

o More than three quarters (77%) agree that since taking 
this course, they think it is more important to be able 
to use technology to succeed in the future; the remainder 
were neutral. 

o Most (85%) thought that computers are a good way to learn 
science material, and a slightly lower percentage (69%) 
reported the same about videodisc systems.. 

o Two thirds (69%) agree that they learn science better 
using a videodisc system, and two thirds (69%) agree 
that they learn science better using a computer. 

o Most (84%) agreed that it is more interesting to learn 
science using a videodicc system, and 88% agreed (54% 
strongly agreed) that it is more interesting to learn 
science using a computer. 

o Three fourths (76%) reported that they like using 

computers in their science class, and a higher percentage 
(85%) would like to use computers in other courses; two 
thirds (69%) like using the videodisc system in their 
science class, and 77% would like to use videodiscs in 
other courses. 

Although the students' attitudes toward science were mostly 
positive, they did not express an interest in pursuing science 
outside the classroom. For example, while more half of the 
students surveyed agreed that science lessons are fun (53%) , 
almost half (46%) were neutral. Almost half (46%) said science 
was interesting to them, one third were neutral, and one fourth 
expressed disinterest. In response to such questions as "I would 
like to belong to a science club," and "I like to read books 
about science," however, most students answered negatively or 
neutrally. 



12 



90.34 



Responses to the attitude toward science questions indicate that: 

o Most (70%) said that they would not like to take another 
science course after this one; almost one fourth (23%) 
were neutral, and one respondent (8%) would like to take 
more science courses. 

o A vast majority (85%) disagreed with the statement, 

"Since taking this course, I am more likely to pursue a 
career in science." One respondent (8%) strongly agreed. 

Some responses, however, did indicate that attitudes toward 
science had become more positive: 

o Nearly one third (30%) said that since they started 
taking this course, they have become more interested 
in science; almost half (46%) were neutral, and about 
one fourth (23%) have not increased their interest. 

o Almost half (46%) agreed that since taking this course, 
they think it is more important to study sc-.ence to 
understand how things work; more than one Uiird (38%) 
were neutral, and 15% disagreed with the statement. 

Did students' knowledge about technology in their environment 
increase? 

Yes. More than half (53%) of the students reported that they had 
noticed more computers being used at school and outside of school 
since taking this course. A small percentage (8%) noticed more 
videodisc systems being used at school, and nearly one third 
(30%) noticed them being used outside of school. Most (77%) 
students agreed that since taking this course, they think it is 
more important to be able to use technology to succeed in the 
future . 

Did teacher use of technology-oriented instructional programs 
increase? 

Yes. In an interview during the spring semester of 1991, the 
science teacher was asked about her use of technology-oriented 
instructional programs. She reported that she used the video 
lessons to instruct other classes in addition to the Introduction 
to Physical Science course. She believes the system is more 
effective in depicting relationships between concepts (e.g., 
volume and density) than a textbook is, and it is easier for 
students to grasp the concepts because of how they are visualized 
on the computer. 
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APPENDIX 

TLTG Student Attitude Survey 
May 1991 

ATTITUDE TOWARD SCIENCE 



Survey Question 


SA 


A 


Hi Responding 
N 


D 


SD 


Science lessons are fun. 


15X 


38X 


46X 


OX 


OX 


Science is interesting to me. 


8% 


38X 


31X 


15X 


8X 


1 would like to bwlong to 
a science club. 


15X 


8X 


8X 


62X 


8X 


1 like to read books about science. 


OX 


8X 


54X 


38X 


OX 


1 lit* to do experiments more 
than read about them. 


46X 


23X 


23X 


8X 


OX 


I would enjoy visiting a 
science museum. 


15X 


54X 


31X 


OX 


OX 


Schools should have more science 
lessons each week. 


8X 


OX 


38% 


54X 


OX 


1 would like to do science 
experiments at home. 


15X 


31X 


31X 


31X 


8X 


It is more interesting to learn 
/science with a computer. 


54X 


23X 


15X 


OX 


8X 


It is more interesting to learn 
science with a videodisc system. 


46X 


38X 


OX 


8X 


8X 


1 would like to be a scientist one day. 


OX 


8X 


OX 


62X 


23X 


I would like to take more science 
courses after this one. 


8X 


OX 


23X 


62X 


8X 


I enjoy watching science programs on TV. 


23X 


23X 


31X 


14X 


8X 


I oet excited about neu thinas in 

. *IV % Cnv 1 % «\J UWW % | ^Crl Villi III 

science class. 


8X 


23X 


38X 


23X 


8X 


I like to find out about new things. 


31X 


31X 


31X 


8X 


OX 


I learn science better using a videodisc. 


23X 


46X 


23X 


OX 


8X 


I learn science better using a computer. 


38X 


31X 


15X 


8% 


8X 


Since taking this course, 1 have become 
more interested in science. 


15X 


15X 


46X 


23X 


OX 


Since taking this course, 1 am more 
likely to pursue a career in science. 


8X 


OX 


8X 


62X 


23X 


Since taking this course, 1 think it is 
more important to study science to understand 
how things work. 


15X 


31X 


38X 


15X 


OX 


SA ■ Strongly Agree A » Agree N * 


Neutral 


D * 


Disagree 


SO » 


Strongly 



N=13 
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ATTITUDE TOWARD TECHNOLOGY 



Survey Question 


SA 


^ % Responding 


D 


so 


I like using computers in my science cless. 


38% 


38% 


15% 


0% 


8% 


I think computers are a go<d way to 
learn science material. 


31* 


54% 


8% 


OX 


ax 


I like using a videodisc system 
in my science class. 


38% 


31X 


15X 


8% 


8X 


I think videodisc systems are a good way 
to learn science material. 


38X 


31% 


15% 


8% 


8X 


I would like to use computers in 
my other courses. 


54% 


31% 


8% 


8% 


OX 


I would like to use a videodisc system 
in my other courses. 


46% 


31% 


15% 


8% 


OX 


i am comToriaoie using cwijjuici »• 


54% 


23% 


23X 


0% 


OX 


I am comfortable using a videodisc. 


31% 


38% 


23% 


8% 


OX 


I enjoy using the videodisc system. 


38% 


31% 


15% 


8% 


ox 


I enjoy using computers. 




38* 


0% 


ox 


ox 


I would rather learn about science 
from a teacher and a textoooK man ay 
using a videodisc system. 


8% 


8% 


15% 


31X 


38* 


I would rather learn about science 
from a teacher and a textbook than oy 
using a computer. 


8% 


8% 


OA 


38X 


38* 


I would enjoy learning more 
about now cwnputers worx. 


62% 


31% 


8% 


OX 


OX 


I would enjoy learning more about 
how videodisc systems work. 


46% 


23% 


23% 


OX 


OX 


Since taking this course, I have noticed 
more computers being used in my school. 


31% 


23% 


31% 


15X 


OX 


4>L{ 4U2a i»m ipsa f K 

Since teicing this course, i nave 
noticed more videodisc systems being 
used in my school. 


8% 


0% 


62% 


23X 


8X 


Since taking this course, I have 
noticed more computers being used 
outside of school. 


15% 


38% 


15% 


23X 


8X 


Since taking this course, I have 
noticed more videodisc systems being 
used outside of school. 


15% 


15% 


38% 


23X 


8X 


Since taking this course, I have become 
more interested in technology as a way 
to learn in school. 


15% 


15% 


38% 


23X 


8X 


Since taking this course, I think it is 
more important to be able to use technology 
to succeed in the future. 


15% 


62% 


23% 


OX 


OX 



SA * Strongly Agree A = Agree N = Neutral D = Disagree SO = Strongly Disagree 

N * 13 



ERIC 



15 
ii 



90.34 
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